Generative AI for consulting: focus on the task not the role

Harvard Business School has published an informative working paper of a study into the impact of generative AI and Large Language Models (LLMs) on the provision of management consulting services, conducted with support of consultants from Boston Consulting Group.

The paper highlights the need to look at the specific tasks that generative AI can enhance rather than at particular roles, introducing the concept of a “frontier” between tasks for which the support provided is productive (“inside the frontier”) and those for which it is non-productive (“outside the frontier”). The paper reports on the results of two theoretical projects, one inside and one outside the frontier, based on the input given by over 750 consultants from Boston Consulting Group and the AI support provided by the ChatGPT LLM.

The inside-the-frontier project focused on a product innovation and development activity involving idea development, market segmentation, writing a press release and an inspirational message about the product. On the other hand, the outside-the-frontier project involved the development of strategic recommendations using quantitative data and interview notes to identify a brand with the greatest potential for growth and to develop actions to achieve this. A more detailed description of the scope of each project is included in Appendix A (numbered pages 40 to 42) of the paper.

The consultants performing these tasks were organised into three groups: one not using AI, one using ChatGPT, and one using ChatGPT as well as receiving instruction on how to make effective use of it.

For the task inside the frontier, the consultants using ChatGPT completed 12.2% more activities, completed them 25.1% more quickly and with a 40% higher level of quality than the consultants not using ChatGPT. The use of ChaptGPT also had a significant impact on the performance of poorer performing consultants whose performance increased by 43%, compared to a performance increase of 17% for higher performing consultants.

However, for the outside the frontier task, consultants using ChatGPT who had also been trained were 24% less likely to produce correct solutions while those using ChatGPT who had not been trained were 13% less likely to produce correct solutions.

The authors of the paper also noted two different styles of using generative AI: Centaurs who establish a clear division between the use of AI and human intervention, and Cyborgs who adopt a more integrated approach.

They also draw a number of important conclusions:

• the best approaches to using AI are not fully understood and need to be deeply examined by scholars and practitioners;
• there is a risk that work “inside the frontier” stops being delegated to junior workers, creating long-term training deficits;
• the big challenge for consultants is “how to navigate the frontier”. Expertise is required to do this successfully, involving a combination of education, on the job training and upskilling

My own conclusion is that the key to successful use of generative AI and LLMs is not what you use, it’s the way in which you use it. Perhaps the GPT in ChatGPT should stand for General Purpose Tool instead of Generative Pre-trained Transformer.

 

Jim Foster, Associate Director, CMCE